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ABSTRACT 

The usability test team examined design problems that limit 
the ability of instructors at the Indiana University to use data entry forms 
on the School of Education Web site. The forms permit instructors to publish 
information about themselves and about courses they teach on the School of 
Education Web site. Faculty and graduate student instructors were asked to 
perform typical tasks with the Web forms under observation. Participants were 
asked to describe what they were thinking and feeling as they attempted to 
perform the task. From these observations, several key design problems were 
identified that prevented or frustrated the participants. Solutions are 
recommended to eliminate these problems. The paper concludes that the 
designers of this Web site should address the following problems in future 
versions of the site. First, the recommended procedure for adding an HTML- 
formatted syllabus to the School of Education Web site fails to produce 
satisfactory results. Designers should find tools that produce reliable 
results, or eliminate this option. The School of Education may need to make 
organizational changes such as employing specialists to assist instructors in 
preparing electronic versions if this feature is desired. Second, the 
Personal Information page causes problems for uses due to its length and 
complexity. At a minimum, the form should be separated into two pages, one 
for required information and one for optional information. In addition, the 
designers should make sure that all multi-line text fields handle carriage 
returns properly. Third, the designers should add navigational elements to 
the Web forms that provide users with indications of what steps they have 
completed and what steps they still need to complete. (Author/AEF) 
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USABILITY TESTING OF THE INDIANA UNIVERSITY EDUCATION FACULTY 

WEB FORMS 

Hakan Tuzun 
Sun Myung Lee 
Charles Graham 
Kirk Job Sluder 

Indiana University Bloomington 



Abstract 

The usability test team attempted to identify design problems that limit the ability of instructors at the 
Indiana University to use data entry forms on the School of Education Web site. The forms permit 
instructors to publish information about themselves and about courses they teach on the School of 
Education web site. We asked faculty and graduate student instructors to perform typical tasks with the 
web forms under observation. We requested that the participants describe what they were thinking and 
feeling as they attempted to perform the task. From these observations, we identified several key design 
problems that prevented or frustrated the participants. We also recommended solutions to eliminate these 
problems. 

Introduction 

Over the past couple of years the use of the World-Wide Web (WWW) has increased dramatically. People 
use the WWW to gather information that informs their day-to-day decision making processes. Organizations of 
every kind are also making information readily available on the Web. The School of Education at Indiana 
University has recently decided to make information about course instructors, and information about classes taught 
in the School of Education available on their public web server (http://education.indiana.edu). Instmctors can add 
information to their listing under the faculty/staff directory. Instructors can also add information about classes they 
teach to the School of Education course catalog. The ability of users to effectively perform tasks with web sites and 
data entry interfaces is a major factor in the total cost of a computer information system. Poor design can add to the 
cost of an information system by reducing efficiency, and adding to instructional and performance support costs 
(Landauer, 1995). Design problems that hinder the ability of instructors to use these forms may also hinder the 
adoption of these forms by the School of Education community, and also add to the hidden costs of adding these 
forms. 

People use computers in order to achieve goals within a situated context (Suchman, 1 987). Usability testing 
involves observing people attempting to achieve specific goals within a specific context. Frick & Boling (1999) 
claim that three factors are considered important in the design of a usability test for a web site: 

1 . The participants must match the intended audience of the web site. 

2. The participants must be given authentic tasks that they may be required to perform as part of their 
use of the web site. 

3. The context in which the test is performed should match the actual work context in which the 
users would perform authentic tasks. 

Methodology 

In this research, we focused our data gathering efforts on the target population of faculty and staff of the 
School of Education. This group also includes graduate students working as Assistant Instructors in the school. The 
following sections will explain the sample population, the research questions, the process of collecting data, and the 
process of debriefing and data analysis. 

Population Sample 

We used a convenience sample in order to try and get a number of participants from our target population 
with medium to low HTML and web skills. We felt that with a small population sample of 8-10 participants, and 
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participants with less web experience would accentuate problems in the web forms interface we were testing. In 
addition to seeking participants with low web skills, we also sought to get participants from many different 
departments. We also tried, without success, to get an even gender representation in our population sample. Table 1 
shows information about the nine participants in our usability tests. 



Table 1. Participants in the Usability Test 



Participant 


Position 


Skill Level 


Department 


Gender 


Participant 1 


Research Faculty 


Low 


Dean’s Office 


Female 


Participant 2 


Faculty 


Low 


Early Childhood 


Male 


Participant 3 


AI 


Medium-low 


Language Education 


Male 


Participant 4 


Faculty 


Low 


Instructional Systems 
Technology 


Male 


Participant 5 


Faculty 


Low 


Educational Leadership and 
Policy Studies 


Male 


Participant 6 


AI 


Medium-low 


Math Education 


Male 


Participant 7 


Faculty 


Low 


Special Education 


Male 


Participant 8 


Faculty 


Low 


Educational Leadership and 
Policy Studies 


Male 


Participant 9 


Faculty 


Low 


Secondary Education 


Male 



Research Questions 

There are two main questions that we investigated as part of this project: 

1 . What are the design problems that prevent or discourage faculty and staff from using the web 
forms to add/update their online profile information? 

2. What are the design problems that prevent or discourage faculty and staff from using the web 
forms to add/update a syllabus link to a course in the School of Education course catalog? 

These are both important questions to address because if the faculty and staff cannot easily update this 
information, then there is a high likelihood that they will not take the time to add/update their personal information 
and course information. This could potentially result in frustration by users seeking information using the school’s 
web pages. 

Data Collection 

We observed the participants performing the following two tasks: 

1. Add/update their personal profiles to the School of Education faculty and staff directories on the 
Web. 

2. Add a syllabus link to a course in the School of Education course catalog on the Web. 

We tried to make the tasks as authentic as possible by 

• Having the participants perform the tasks in the environments where they would normally 
perform the tasks (e.g., their offices or public computing labs). 

• We contacted the participants ahead of time and asked them to have a copy of their 
syllabus (or know the link if already online) ready to use in the testing. This allowed the 
ones who came prepared with their own syllabi, to post their syllabi to the web for an 
actual course. (We also had a copy of a syllabus on disk in MS Word format that a 
couple of individuals who had forgotten used. In these cases, they posted to a test course 
R999 that was set up by the Web director.) 

We also swapped the order that different participants were asked to do the two tasks in order to try and 
eliminate overlooking problems that might occur by doing the tasks in different orders. 

We asked the participants to “think aloud” while they were performing the tasks. Meanwhile, the observers 
took notes of difficult ies and problems they were having. We also timed each of the tasks. 

Data Analysis 
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After all of the usability testing observation sessions had been completed, the four members of our project 
team met to debrief and analyze our findings. We grouped similar problems together and recorded a short 
description of the problem along with the number of participants that reported the problem. We then ranked the 
problems according to the following criteria: 

1. Most critical: Problem prevented the subject from accomplishing the desired task or the subject 
reported high levels of frustration in regards to this problem. 

2. Critical: Problem resulted in confusion or misdirection that did not prevent the subject from 
accomplishing the desired task. 

3. Not-critical: Problem was reported as confusing or undesirable but there is no urgent need to fix 
this problem to insure functionality. 

4. Low-priority: Problem should be examined on a case-by-case basis. 

We then linked the problems to specific stages in the process at which the problem occurred. We also provided 
recommendations for fixing the problem, as well as recommendations for re-designing the site as a whole. 

Findings and Recommendations 

Task 1: Add/Update Profile 

In our usability test we asked the participants to add or update their personal profile. The personal profile 
is published with the staff and faculty directory, and is linked to entries in the course catalog. Figure 1 shows the 
task of adding/updating a profile broken down into six sequential steps. The web forms interface consists of three 
separate pages. The first page (Demographics) asks for the instructor’s name and email address. The second page 
(Contact Info) asks for office location and telephone number. The third page (Personal Info) asks for a large amount 
of information including job title, department, home page and optional fields for more information and links to other 
web sites. 

Most participants experienced problems on the Personal Info page. Participants reported frustration with 
the large size of the page and the variety of information requested. Some participants also reported problems 
interpreting instructions and anticipating how the information they entered would be displayed. 



Step 1 




Step 2 




Step 3 




Step 4 




Step 5 




Step 6 




Getting there 




Demographics 




Contact info 




Personal info 




Preview 




Done 



























Figure 2. The Six Steps Required to Complete the Task of Adding/Updating a Profile 



Step 1: Getting to the Profiles Page 

This section of the report deals with challenges that the users had in finding the web forms that they could 
use to add or update their personal profiles. Table 2 outlines our findings and recommendations for improvement of 
this step in the process. 



Table 2. Findings and Recommendations for Getting to Profile Web Forms 



Findings (Observations) 


Rating 


Recommendation 


Three participants experienced some difficulties 
in finding the first profile web form page. These 
individuals followed the path that took them to 
the Faculty and Staff Directory. They would go 
to their existing profile and from that page (the 
old profile formats) there was not a link to 
update their individual profile. Although there 
was a link to update profiles from the Faculty 
and Staff Directory page it was not easy for 
these participants to locate. 


1 


The impact of this problem may be reduced by making 
the link on the Faculty and Staff Directory more 
obvious. Currently the only word that is linked is the 
word "here” that is embedded in the middle of a couple 
of lines of text. The link text could be changed to 
something like "Add/Update Your Profile" and placed 
on its own line. 


Three participants expressed uncertainty about 


3 


Try re-writing documentation to eliminate ambiguity. 
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which password they should use in order to 
update their profiles. Only one participant 
actually typed in the wrong password and had to 
go back and re-enter another password. 



Unfortunately this problem needs to be addressed by 
University Information Technology Services and not 
the designers of the web forms. 



Step 2: Adding/Updating Demographic Information 

This section of the report deals with challenges that the users had in adding or changing their name and 
email address. Table 3 outlines our findings and recommendations for improvement of this step in the process. 



Table 3. Findings and Recommendations for Adding Demographic Information 



Findings (Observations) 


Rating 


Recommendation 


Two participants discovered that the middle 
name did not appear in the final profile. One of 
the individuals uses his middle name as his 
preferred name, and ended up getting around the 
system by placing his middle name in the first 
name slot. 


2 


Don't ask for a name that will not be included on the 
profile page. Both of the following solutions might be 
acceptable: 

• Include all three names in the profile page. 

• Have a preferred name slot and a last name 
slot. 

• Have one text blank labeled "Full Name to 
appear in profile" 


Two of the participants were confused by the 
wording on the submit button on this page 
which read, "Save the data in this form and add 
your office location information." They didn't 
realize that the "and add your office location 
information" part of the text was referring to 
what they would do on the next page. 


2 


Our recommendation for this option is to add a wizard 
style navigation header and footer to each of the profile 
pages. (This recommendation will also address several 
other findings from the usability test.) For details on 
this recommendation, see the General 
Recommendations for Add/Update Profiles Pages 
section of this report. 

If the wizard style navigation recommendation is not 
taken, the wording of the button could be changed to 
something like, "Save the demographic info and go 
onto the next step." 



Step 3: Adding/Updating Contact Information 

This section of the report deals with challenges that the users had in adding office location and contact 
information to their personal profile. Table 4 outlines our findings and recommendations for improvement of this 
step in the process. 



Table 4. Findings and Recommendations for Adding Contact Information 



Findings (Observations) 


Rating 


Recommendation 


One individual didn't realize that when he 
updated his room number he also had to click on 
the corresponding radio button. (Incidentally, if 
the radio button is selected without entering a 
room number in the text box, you are taken to a 
blank screen which says, “Your form was 
Processed” and nothing else.) 


1 


The form could be simplified by having a box in which 
the room number could be placed and instructions to 
leave the box empty if they don't have a room. 

The current radio buttons could be kept and JavaScript 
could be used to select the appropriate radio button 
when a number is entered in the text box. 



Step 4: Adding/Updating Personal Information 

This section of the report deals with challenges that the users had adding personal information about 
themselves and what they do in the School of Education. Table 5 outlines our findings and recommendations for 
improvement of this step in the process. 
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Table 5. Findings and Recommendations for Adding Personal Information 



Findings (Observations) 


Rating 


Recommendation 


Five of the participants reported skipping the 
optional “More about me” fields because of 
perceived complexity of the instructions for 
entering HTML. They were not able to add 
information to their profiles that they wanted to 
because they felt that it would be too complex 
for them to do it. 


1 


Simplify the task by using formatted plain text only. 
OR 

Give radio button choice for each text box indicating 
“Plain Text (default)” or “HTML” like in AltaVista 
Forum (AVF) 


Participants didn’t seem to understand if text 
boxes would be WYSIWYG or HTML. In one 
case a participant entered multiple titles 
separated by a carriage return and was surprised 
to see them squashed together on the final 
profile. (There are inconsistencies - for example 
the "What You Do" section converts completely 
to HTML with no hard carriage returns while 
the "More About Me" section allows links to be 
added but also recognizes carriage returns.) 


1 


Simplify the task by using formatted plain text only. 
OR 

Give radio button choice for each text box indicating 
“Plain Text (default)” or “HTML” like in AVF 
OR 

Use the server-side CGI script to check text block for 
any HTML tags. If there are tags format as HTML 
otherwise format as plain text. 


Three participants were a little confused by the 
text box labeled “Name of your office/unit. At 
least one filled in the name of his office even 
though he had checked the box for the office 
above. The indentation of the fields relative to 
the other options seemed confusing to the 
participants. People tended not to read the * 
instructions carefully. 


2 


Align all choices equally. Have a checkbox directly 
below the others followed by a blank text box for them 
to enter their office/unit if it is different. A * could be 
placed at the end of the text box and another * could be 
placed below with instructions about how to enter the 
URL for the “other” unit. 


Two participants expressed uncertainty about 
what to put into the section labeled "What You 
Do." The concern was whether this was just for 
faculty /staff job responsibilities or if it could 
contain other kinds of information too. 


2 


This problem could easily be solved by providing more 
explicit guidelines about what should go in this area. 
Also, being able to see an example of a typical profile 
might allow them to see where it would be most 
appropriate to stick different pieces of information 
about themselves. 


It was our observation that participants were 
overwhelmed with the amount of information 
being requested on this page. 


2 


This problem could be solved by using the wizard style 
navigation (see description in section General 
Recommendations for Add/Update Profile Page). 

Using this system this step could easily be divided into 
several more simple steps. 



Step 5: Previewing and Publishing a Profile 

This section of the report deals with challenges that the users had in previewing and publishing their 
profile. Table 6 outlines our findings and recommendations for improvement of this step in the process. 



Table 6. Findings and Recommendations for Previewing and Publishing a Profile 



Findings (Observations) 


Rating 


Recommendation 


Three of the participants considered clicking the 
"Update this profile" link that is embedded into 
the profile preview instead of the publish 
preview button. This generally happened 
because they were scrolled down to the middle 
of the page where they could not see the buttons 
and the link text sounded like it would do what 
they wanted to do. 


1 


To solve this problem we suggest that changing the 
wording of the link text from "Update this profile" to 
something like "Start the update profile wizard" or "Go 
to the Update Profile Pages." We feel that a different 
wording like this will be harder to confuse with 
publishing or actually saving the updates. 


Three participants in the usability test did not 


1 


Doing something to make the reload reminder obvious, 
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remember to click the reload button after going 
back to make changes to their profiles, even 
though the first line of instructions reminded 
them too. In general, they read the instruction 
block the first time they arrived at the page but 
did not re-read it on subsequent visits. 




even without reading the instruction block over, could 
solve this problem. A couple of possibilities might be 
changing the color of the word "reload" and making it 
bold. Or the word reload could be made to blink on 
and off. 

OR 

A server-side solution might be to write the temporary 
profile to a file with a different name each time - 
perhaps <timestamp>.html. 

When the user publishes the profile the temporary file 
is written and deleted. 

OR 

The tradeoff might make using a meta-tag that 
automatically forces the page to be reloaded every time 
worth it. 


Several participants commented that it would 
have been helpful to be able to see examples of 
profiles or even preview their own profiles 
during the process instead of having to wait until 
the very end and then go back to make changes 
at the appropriate step. 


2 


This problem could be taken care of by implementing 
the solution that is mentioned under wizard style 
navigation section later in the report. Under this 
system, the users could preview their profile at any step 
through the process. 


One individual didn't readily see the information 
entered in the "More About Me" fields on the 
profile preview. He finally found them below 
the map and said that he felt like they should 
have belonged above the map. 


3 


Move the link to the “More about me” entries from the 
middle of the demographic information to a more 
prominent location. 



Step 6: Post-publishing steps 

This section of the report deals with challenges that the users had with navigational steps required after the 
profile is published on the web. Table 7 outlines our findings and recommendations for improvement of this step in 
the process. 



Table 7. Findings and Recommendations for Post-Publishing Steps 



Findings (Observations) 


Rating 


Recommendation 


One person exited the web browser directly after 
publishing the profile because he was following 
the red highlighted text at the bottom of the page 
literally. Then he started his web browser right 
back up. 


4 


The wording of the message could be changed to not 
imply that the browser should be closed right then, but 
that it should be closed before they leave the machine. 


One person indicated that he felt like there were 
too many link options on the last page. He felt 
that it was confusing. 


4 


We are not sure that this is a problem that needs fixing. 



General Recommendations for Add/Update Profile Pages 

We have one general recommendation for the profile submission forms. This is a general recommendation 
because it addresses several of the findings observed in the usability testing and mentioned earlier in the document. 
Our general recommendation is to change the navigation system to a "wizard style navigation". What we mean by 
this is explained in the following section. 

Wizard Style Navigation 

A "wizard" is a tool that helps to step a person through a linear process by displaying the current position in 
the process as well as providing controls to move forward or backwards in the process. We think that including a 
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wizard navigation header and footer on each page would be an ideal way to address many of the issues mentioned 
earlier in the findings. The wizard navigation would graphically outline the steps and indicate where the person is in 
the overall process. It should contain buttons to allow the user to progress forward through the steps as well as 
backwards through the steps. Ideally, the navigation bar would also have a preview button and a publish button 
which would allow the user to preview or publish the current state of the profile at any time in the process. 

Task 2: Add/Update Syllabus Link to Course Catalog 

This task tests web forms that allow instructors to publish information about a course they teach on the 
School of Education course catalog. The web forms give instructors four options for entering a course description or 
syllabus: 

1. Enter a syllabus as HTML- formatted text. 

2. Enter a syllabus as plain text with carriage returns. 

3. Enter a link to an existing syllabus on the WWW. 

4. Enter a link to Oncourse, a web-based academic conferencing system. 

Seven participants tried the HTML path. One participant tried the Plain Text path and two tried the 
syllabus link path. No individuals tried the Oncourse path. One participant tried multiple paths in order to achieve 
good results. 

The most critical problem we found was that out of seven participants who attempted to enter HTML- 
formatted text, six found that following the instructions did not produce the desired results. The instructions on the 
web form direct the participants to convert their syllabus from a word processor document to HTML text using 
Microsoft Word 97, then paste the resulting HTML-coded text into a web form text box. In most cases Word 97 
failed to produce readable HTML formatted text. 




Figure 3. The Six Steps for Adding an Entry to the Schedule of Classes 



Step 1: Getting To the Add or Update a Syllabus Page 

This section of the report deals with challenges the users had finding the web forms that permit them to add 
a course entry to School of Education web site. Table 8 outlines our findings and recommendations for 
improvement of this step in the process. 



Table 8. Findings and Recommendations for Getting to the Add a Course Web Form 



Findings (Observations) 


Rating 


Recommendation 


The link text (on page updates.html) indicates 
that you can add a syllabus link but not update a 
syllabus link. 


2 


Change the link text to say "Add/Update . . 


The link text to enter the forms from the course 
catalog is somewhat confusing. It states 
"Add/Modify your course web site." 


2 


The text should be changed to read something link 
"Add/Modify syllabus link for this course" or 
something similar. In addition the text at the top of the 
course catalog table should be made clearer by 
changing it to "Syllabus Link(s)" instead of just 
"Link(s)" 



Step 2: Entering a Course Number and Choosing a Format 

This section of the report deals with challenges the users had entering a course number and choosing a 
syllabus format. Table 9 outlines our findings and recommendations for improvement of this step in the process. 
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Table 9. Findings and Recommendations for Entering a Course Number and Choosing a Format 



Findings (Observations) 


Rating 


Recommendation 


Four participants typed a lower case letter first 
for the course number. 


2 


Use server-side CGI to set the case of the letter rather 
than forcing user to do it. 


When the participants got the error message for 
typing in a lower case letter or not selecting a 
radio button the error message is incorrect and 
says "Click the back button and enter the 
filename in CAPITAL letters." 


2 


Check all error messages and make sure that they are 
appropriate for the error. The two errors mentioned 
should have different error messages and the message 
for the first error should read "Course Number" instead 
of "filename." 



Step 3. a: Adding an HTML-Formatted Syllabus 

This section of the report deals with challenges the users had adding an HTML-formatted syllabus to the 
School of Education web site. Table 10 outlines our findings and recommendations for improvement of this step in 
the process. 



Table 10. Findings and Recommendations for Adding an HTML Formatted Syllabus 



Findings (Observations) 


Rating 


Recommendation 


Six participants followed the instructions to 
convert a word-processed document to HTML 
and didn't get results that they were satisfied 
with. (Tables and tab-formatted documents 
notably had problems and participants didn't 
have skills to make changes.) 


1 


See general recommendations below. 


Five of the participants complained about being 
forced to use MSWord. Their word processor of 
choice was WordPerfect. 


1 


Same as above. 


The forms don't allow the user to edit a syllabus 
that has been added. To make one change to the 
syllabus they must edit their Word document 
and go through the whole conversion process 
again. (This can be painful for just a spelling 
change or something.) 


1 


Allowing to upload syllabi in their original format 
would solve this problem. 

OR 

Load the syllabus HTML text into the text box when a 
syllabus already exists. 


Five individuals commented on the reference to 
the "Title Box below" when the title box was 
really above 


2 


Change the text to read "above" instead of "below." 


Following the HTML instructions was difficult 
because the users had to switch back and forth 
between the word processing window and the 
browser window. Each time the users had to re- 
find where they were in the process. (One 
individual skipped the step of changing the word 
processor to "view html source" and therefore 
ended up in the end with a bunch of words all 
scrunched together - spaces removed.) 


2 


Suggest in the instructions that the user print out the 
instructions to reduce the chances of skipping steps. 



Step 3.b: Adding a Plain Text Formatted Syllabus 

This section of the report deals with challenges the users adding a plain text syllabus to the School of 
Education web site. Table 1 1 outlines our findings and recommendations for improvement of this step in the 
process. 



Table 11. Findings and Recommendations for Adding a Plain Text Formatted Syllabus 



Findings (Observations) 


Rating 


Recommendation 


One participant pasted plain text copied from a 


2 


Strengthen wording in instructions to indicate that most 
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word -processing document into textbox but was 




text formatting will be lost. 


unsatisfied with results. 







Steps 3.c: Adding a Link to an Existing Syllabus Elsewhere on the WWW 

This section of the report deals with challenges the users had adding a hypertext link from the School of 
Education course catalog to an existing syllabus elsewhere on the WWW. Table 12 outlines our findings and 
recommendations for improvement of this step in the process. 



Table 12. Findings and Recommendations for Adding a Link to an Existing Syllabus Elsewhere on the WWW 



Findings (Observations) 


Rating 


Recommendation 


Participant was unable to find syllabus 
published on School of Education web server 
prior to installation of new web forms. 


3 


Provide suggestions for finding URL of published 
syllabus. Also consider providing links to Indiana 
University web search tool. 



Steps 3.d: Adding a Link to Oncourse 

This section of the report deals with challenges the users had adding a hypertext link from the School of 
Education course catalog to Oncourse, an online academic information system developed by Indiana University 
Information Technology Services. No participants selected this option. 

Step 4: Adding Additional Notes to the Course Entry 

This section of the report deals with challenges the users had adding additional notes to the course entry. 
Table 13 outlines our findings and recommendations for improvement of this step in the process. 



Table 13. Findings and Recommendations for Adding Additional Notes to a Course Entry 



Findings (Observations) 


Rating 


Recommendation 


Four of the users did not even enter anything in 
the notes block because they were just not sure 
what kind of information should go there. One 
individual put several paragraphs about himself 
in the notes area. 


2 


The instructions should be changed to indicate what 
kind of information should be included in the notes. 
AND 

An example or preview might help. 



Step 5: Viewing Summary Information 

This section of the report deals with challenges the users had interpreting summary information provided 
about a course entry. No problems were observed with this step. 

Step 6: Post-Publishing Steps 

This section of the report deals with challenges the users had viewing updated course entries after they 
were published. Table 14 outlines our findings and recommendations for improvement of this step in the process. 



Table 14. Findings and Recommendations for Post Publishing Steps 



Findings (Observations) 


Rating 


Recommendation 


Participant went back through web forms to 
make changes. Participant then tried to view 
changes but failed to reload web page. 


2 


Add tag to web pages to force the course schedule and 
syllabus to reload if possible. 



General Recommendations for Adding or Updating a Course Entry to the School of Education Course 
Catalog 



Converting a word -processing document to HTML text caused serious problems for all but one of the 
individuals that attempted the process. This procedure resulted in four different types of problems affecting a large 
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percentage of our participants. We do not feel we have enough information to properly recommend a solution to 
this problem. Possible solutions may include providing the ability to upload word processing documents, 
conversion of word processing documents to Adobe PDF, or providing technical assistance. 

Participants also reported problems with the instructions provided on the web forms and with labels of text 
fields. We recommend that the instructions and descriptive text be edited to correct errors and reduce ambiguity. 

Limitations 

Three of the main limitations to our findings involve the selection of our participants: 

1 . Only one of our participants was female. 

2. Only two of our participants were Associate Instructors. 

3. We did not recruit participants that had high levels of computer experience. 

We attempted to recruit more women and associate instructors. However we were unable to arrange test 
sessions with most of the women and associate instructors we contacted. It is possible that having greater 
representation in both of these groups may have added insights to the information we gathered. We also did not 
recruit participants with high levels of computer experience. We deliberately selected a convenience sample of 
individuals with low to moderate computer experience in order to maximize observation of design problems 
experienced by novice users. As a result, we don’t make any generalizations regarding the ability of instructors in 
the School of Education to use these forms. 

Additionally, none of the people that we did the usability testing with had done either of the tasks before. 
Therefore, our tests did a good job of testing what a novice to the system would do in adding information to the 
system, but didn't explore what challenges would be faced by someone who had already entered data andjust 
wanted to make a small change (e.g., a new phone number, link, or spelling correction/date change to syllabus). We 
have a hunch that there may be additional problems that surface when this aspect of the system is tested. 

Some other problems came out during testing. None of the participants tried to add an Oncourse syllabus to 
the course catalog. Although the process is very similar to adding a syllabus link, some testing of this should be 
done to ensure usability. Several participants also reported that they would have abandoned some steps in the 
process if they had not been participating in a usability study. Participants reported that being observed encouraged 
them to tackle problems that would have led them to give up. 

Conclusion 

We found multiple problems with the design of the web forms that hindered or prevented our participants 
from successfully completing these tasks. Re-designing the web forms in order to reduce or eliminate these 
problems is in our opinion essential to ensuring the adoption of these web forms within the School of Education. 

The designers of this web site should address the following problems in future versions of the web site: 

1. The recommended procedure for adding an HTML- formatted syllabus to the School of Education 
web site fails to produce satisfactory results. The designers should find tools that produce reliable 
results, or eliminate this option. The School of Education may need to make organizational 
changes such as employing specialists to assist instructors in preparing electronic versions if this 
feature is desired. 

2. The Personal Information page causes problems for uses due to its length and complexity. At a 
minimum the form should be separated into two pages, one for required information and one for 
optional information. In addition the designers should make sure that all multi-line text fields 
handle carriage returns properly. 

3. The designers should add navigational elements to the web forms that provide the user with 
indications of what steps they have completed and what steps they still need to complete. 
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